Image retrieval system, information processing apparatus, image retrieval method, and non-transitory computer readable medium

ABSTRACT

An image retrieval system includes an obtaining unit that obtains a query image, a specifying information extracting unit that extracts, from the query image, specifying information which specifies an image group as a retrieval target, a feature information extracting unit that extracts, from the query image, feature information to be used in image retrieval processing, and a retrieval unit that performs the image retrieval processing on retrieval target images specified by the specifying information, using the feature information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 fromJapanese Patent Application No. 2013-172375 filed Aug. 22, 2013.

BACKGROUND

(i) Technical Field

The present invention relates to an image retrieval system, aninformation processing apparatus, an image retrieval method, and anon-transitory computer readable medium.

(ii) Related Art

When performing image retrieval processing, the processing amount of theimage retrieval processing is reduced by selecting, as retrievaltargets, only the retrieval target images corresponding to a queryimage, instead of selecting all the registered retrieval target images.

SUMMARY

According to an aspect of the invention, there is provided an imageretrieval system including: an obtaining unit that obtains a queryimage; a specifying information extracting unit that extracts, from thequery image, specifying information which specifies an image group as aretrieval target; a feature information extracting unit that extracts,from the query image, feature information to be used in image retrievalprocessing; and a retrieval unit that performs the image retrievalprocessing on retrieval target images specified by the specifyinginformation, using the feature information.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the present invention will be described indetail based on the following figures, wherein:

FIG. 1 illustrates the overall configuration of an image retrievalsystem according to a first exemplary embodiment;

FIGS. 2A through 2C illustrate the overview of an EMM;

FIG. 3 is a block diagram illustrating the hardware configuration of auser terminal according to the first exemplary embodiment;

FIG. 4 is a block diagram illustrating the hardware configuration of animage retrieval server according to the first exemplary embodiment;

FIG. 5 illustrates an exemplary configuration of an image retrievaldatabase according to the first exemplary embodiment;

FIG. 6 is a functional block diagram illustrating a user terminal andthe image retrieval server according to the first exemplary embodiment;

FIG. 7 illustrates a sequence diagram illustrating processing executedby the image retrieval system according to the first exemplaryembodiment;

FIG. 8 is a flowchart illustrating the flow of registration processingaccording to the first exemplary embodiment;

FIGS. 9A through 9C illustrate a specific example of registrationprocessing according to the first exemplary embodiment;

FIG. 10 is a flowchart illustrating the flow of retrieval processingaccording to the first exemplary embodiment;

FIGS. 11A through 11C illustrate a specific example of retrievalprocessing according to the first exemplary embodiment;

FIGS. 12A through 12C illustrate a method by which an image retrievalsystem extracts specifying information according to a second exemplaryembodiment;

FIG. 13 illustrates a specific example of retrieval processing accordingto a variation (1) of the second exemplary embodiment;

FIG. 14 is a flowchart illustrating the flow of retrieval processingaccording to a variation (2) of the second exemplary embodiment;

FIGS. 15A and 15B illustrate a specific example of retrieval processingaccording to the variation (2) of the second exemplary embodiment;

FIG. 16 illustrates the overall configuration of an image retrievalsystem according to a variation 1;

FIG. 17 illustrates an exemplary configuration of an apparatusmanagement table according to the variation 1;

FIG. 18 is a sequence diagram illustrating the flow of retrievalprocessing according to the variation 1;

FIG. 19 is a sequence diagram illustrating the flow of retrievalprocessing according to a variation 3;

FIGS. 20A through 20D illustrate specific examples of registrationprocessing and retrieval processing according to a variation 5; and

FIG. 21 is a sequence diagram illustrating the flow of retrievalprocessing according to the variation 5.

DETAILED DESCRIPTION

Exemplary embodiments of the present invention will be described withreference to the accompanying drawings.

First Exemplary Embodiment

FIG. 1 illustrates the overall configuration of an image retrievalsystem 1 according to a first exemplary embodiment. As illustrated inFIG. 1, the image retrieval system 1 is an information processing systemthat performs image retrieval, and includes a user terminal 10, an imageretrieval server 20, and a registration terminal 30. Although only oneuser terminal 10 and one registration terminal 30 are illustrated inFIG. 1, there are actually two or more user terminals 10 and two or moreregistration terminals 30. The user terminal 10, the image retrievalserver 20, and the registration terminal 30 connect to a communicationnetwork 100 so as to communicate with each other. The communicationnetwork 100 may be any types of communication networks, including theInternet, for example.

The user terminal 10 is a communication terminal having an imagecapturing function (a camera function). The user terminal 10 captures animage of a printed material 40 of FIG. 1, and transmits a query imagerequesting an image retrieval to the image retrieval server 20. Thequery image is image data serving as a retrieval key of image retrievalexecuted by the image retrieval server 20, and is data representing, inthe form of an image, a request to the image retrieval server 20 forexecution of image retrieval. The image retrieval server 20 is an imageprocessing apparatus that retrieves an image (a retrieval target imagedescribed below) similar to the query image received from the userterminal 10. The printed material 40 is a medium (paper medium) on whichimage elements such as text and photographs are printed. Examples of theprinted material 40 include flyers, brochures, and magazines. On theprinted material 40, a marker 41 called an embedded media marker (EMM)is printed on the image element. The marker 41 is a mark indicating thata digital content is associated with the image element of the printedmaterial 40. The marker 41 is printed in a relatively light color so asnot to be an obstacle for the user to visually recognize the imageelement of the printed material 40. The registration terminal 30 is acommunication terminal used by, for example, a content provider, andregisters information on EMM and information on digital contents in theimage retrieval server 20.

In this example, the user terminal 10 is a smart phone. However, theuser terminal 10 may be any other types of communication terminals thathave an image capturing function, such as mobile phone terminal, tabletterminal, notebook computer, personal digital assistant (PDA), andportable game device. Examples of the registration terminal 30 include apersonal computer and other types of communication terminals.

FIGS. 2A through 2C illustrate the overview of an EMM.

FIG. 2A is a schematic diagram of the marker 41. As illustrated in FIG.2A, the marker 41 includes a feature boundary 411, an anchor point 412,and a media icon 413. The feature boundary 411 is represented by animage of a circular line, and encircles an image element of the printedmaterial 40. As illustrated in FIG. 2B, when capturing an image of theprinted material 40, the user of the user terminal 10 operates the userterminal 10 such that the entire marker 41 is located within the imagingrange. The anchor point 412 is location information that identifies thelocation of the image element to which a digital content is linked. Themedia icon 413 is an icon that identifies the type of the digitalcontent. In this example, the media icon 413 is an icon imagerepresenting a video camera. This icon image indicates that the digitalcontent is a moving image content. In this case, as illustrated in FIG.2C, the user terminal 10 having captured the image of the marker 41displays a moving image C by reproducing a moving image content providedfrom the image retrieval server 20. A moving image content is a digitalcontent related to an image element of the printed material 40.

The digital content associated with the marker 41 may be a still imagecontent, a Web content, or other types of digital contents.

FIG. 3 is a block diagram illustrating the hardware configuration of theuser terminal 10. As illustrated in FIG. 3, the user terminal 10includes a controller 11, a user interface (UI) unit 12, a communicationunit 13, an image capturing unit 14, and a storage unit 15.

The controller 11 includes a processor having a central processing unit(CPU), a read only memory (ROM), and a random access memory (RAM). TheCPU controls each unit of the user terminal 10 by reading a controlprogram stored in the ROM or the storage unit 15 to the RAM andexecuting the read control program. Further, the controller 11 includesan image processing circuit, which is, for example, an applicationspecific integrated circuit (ASIC), and performs image processing usingthe image processing circuit. The UI unit 12 is an operation displaythat provides a graphical user interface (GUI), and includes a displaypanel that displays an image on a display surface, and a touch screenthat is disposed on the display surface and is operated by the user.

The user terminal 10 may include other operation units that receiveoperations from physical keys and the like, and may have a function thatreceives voice input operations.

The communication unit 13 includes an interface for performingcommunications by connecting to the communication network 100. The imagecapturing unit 14 is an image capturing device that includes an imagingelement such as a charge coupled device (CCD), and generates a capturedimage (a still image or a moving image) by capturing an image. An imagecapturing lens (not illustrated) of the image capturing unit 14 isprovided on the back side of the user terminal 10. The storage unit 15includes a storage device such as an electronically erasable andprogrammable ROM (EEPROM) and a flash memory, and stores the operatingsystem (OS) and other programs that are executed by the controller 11.

FIG. 4 is a block diagram illustrating the hardware configuration of theimage retrieval server 20. As illustrated in FIG. 4, the image retrievalserver 20 includes a controller 21, a communication unit 22, and astorage unit 23.

The controller 21 includes a processor having a CPU, a ROM, and a RAM.The CPU controls each unit of the image retrieval server 20 by reading acontrol program stored in the ROM or the storage unit 23 to the RAM andexecuting the read control program. Further, the controller 21 includesan image processing circuit, which is, for example, an ASIC, andperforms image processing using the image processing circuit. Thecommunication unit 22 is an interface for performing communications byconnecting to the communication network 100. The storage unit 23 is astorage device including a hard disk device, for example, and stores aprogram that is executed by the controller 21. The storage unit 23further includes an image retrieval database 231.

Note that although there is an element “SPECIFYING INFORMATION TABLE”denoted by the dashed-line box in FIG. 4, a specifying information table232 is an element of a second exemplary embodiment described below, andis irrelevant to this exemplary embodiment.

FIG. 5 is a diagram illustrating the configuration of the imageretrieval database 231.

As illustrated in FIG. 5, the image retrieval database 231 is a databasein which specifying information, image identifiers, local featureamounts (an example of feature information), and content information areregistered in association with each other.

The specifying information is information that specifies an image groupas a retrieval target. This image group is a collection of retrievaltarget images, and is a part of the all the retrieval target images thatare retrieval targets of the image retrieval server 20. In this example,each content provider that provides digital contents associated withretrieval target images corresponds to different specifying information.That is, in this exemplary embodiment, the specifying information is,but not limited to, an identifier that uniquely identifies the categoryof a retrieval target image.

The image identifier is an identifier that uniquely identifies theretrieval target image. The local feature amount represents the amountof features of a retrieval target image. More specifically, the localfeature amount is the amount of local features of each of plural featurepoints. In this example, the local feature amount is the scale invariantfeature transform (SIFT) feature amount. The SIFT feature amount is thelocal feature amount obtained by determining a representative luminancegradient direction of a pixel, generating a luminance gradient histogramwith reference to the determined direction, and representing a featureby using a multidimensional vector (for example, a 128-dimensionalvector). The SIFT feature amount is well known in the art as described,for example, in Nagahashi, Fujiyoshi, and Kanade, “Object TypeClassification Using Structure-Based Feature Representation”, IEEJ,Technical Meeting on Systems and Control, pp 39-44, January 2007, andtherefore will not be described herein in detail.

The content information is information indicating a digital contentprovided from the image retrieval server 20 to the user terminal 10. Inthis example, the content information is information on the accessdestination, such as a uniform resource identifier (URI) indicating thelocation where the digital content is stored. However, the contentinformation may be information representing the digital content itself.

In the manner described above, in the image retrieval database 231, thelocal feature amounts extracted from a retrieval target image andcontent information indicating the digital content associated with theretrieval target image are registered in association with each other foreach specifying information.

FIG. 6 is a block diagram illustrating the functional configuration ofthe user terminal 10 and the image retrieval server 20. As illustratedin FIG. 6, the image retrieval server 20 executes a program, and therebyrealizes functions corresponding to a retrieval target image receivingunit 201, a feature information extracting unit 202, a specifyinginformation extracting unit 203, a registration processing unit 204, aquery image receiving unit 205, a retrieval unit 206, and a contentinformation output unit 207. The user terminal 10 executes a program,and thereby realizes functions corresponding to a captured imageobtaining unit 101, a query image transmitting unit 102, and a contentinformation receiving unit 103.

Note that although there is an element “IMAGE PROCESSING UNIT” denotedby the dashed-line box in FIG. 6, an image processing unit 208 is afunctional element of a variation (2) of a second exemplary embodimentdescribed below, and is irrelevant to this exemplary embodiment.

The retrieval target image receiving unit 201 of the image retrievalserver 20 receives and obtains a retrieval target image, in response toa registration request of the retrieval target image from theregistration terminal 30.

The feature information extracting unit 202 extracts local featureamounts of feature points of a retrieval target image provided from theretrieval target image receiving unit 201. The feature informationextracting unit 202 also extracts local feature amounts of a query imageprovided from the query image receiving unit 205.

The specifying information extracting unit 203 extracts specifyinginformation from the retrieval target image provided from the retrievaltarget image receiving unit 201. The specifying information extractingunit 203 also extracts specifying information from the query imageprovided from the query image receiving unit 205.

The registration processing unit 204 performs registration processingfor registering data on the retrieval target image in the imageretrieval database 231. More specifically, the registration processingunit 204 registers the local feature amounts extracted from theretrieval target image by the feature information extracting unit 202,in association with the specifying information of the retrieval targetimage extracted by the specifying information extracting unit 203, inthe image retrieval database 231. In this step, the registrationprocessing unit 204 registers content information indicating a digitalcontent that is obtained from the registration terminal 30, inassociation with the specifying information and the local featureamounts, in the image retrieval database 231.

The captured image obtaining unit 101 of the user terminal 10 obtains animage captured by the image capturing unit 14.

The query image transmitting unit 102 transmits, to the image retrievalserver 20, a query image for image retrieval, on the basis of thecaptured image obtained by the captured image obtaining unit 101.

The query image receiving unit 205 (an example of an obtaining unit) ofthe image retrieval server 20 receives and obtains the query imagetransmitted from the query image transmitting unit 102.

The retrieval unit 206 performs retrieval processing on retrieval targetimages specified by the specifying information, using the local featureamounts. The retrieval unit 206 retrieves a retrieval target imagesimilar to the query image transmitted from the query image transmittingunit 102, on the basis of the image retrieval database 231. Theretrieval unit 206 obtains, from the image retrieval database 231, localfeature amounts associated with the specifying information extracted bythe specifying information extracting unit 203, and compares theobtained local feature amounts with the local feature amounts of thequery image extracted by the feature information extracting unit 202 soas to determine a corresponding point for each feature point. Theretrieval unit 206 retrieves a retrieval target image similar to thequery image on the basis of the determined corresponding points, andidentifies the image identifier of the retrieved retrieval target image.

The content information output unit 207 (an example of an output unit)outputs content information corresponding to the retrieval target imageretrieved by the retrieval unit 206 to the user terminal 10. The contentinformation output unit 207 obtains, from the image retrieval database231, the content information which is associated with an imageidentifier of the retrieval target image retrieved by the retrieval unit206. Then, the content information output unit 207 transmits and outputsthe content information to the user terminal 10.

The content information receiving unit 103 of the user terminal 10receives the content information output from the content informationoutput unit 207.

FIG. 7 is a sequence diagram illustrating the flow of processingexecuted by the image retrieval system 1.

When the user performs an operation for requesting registration of aretrieval target image in the image retrieval server 20, theregistration terminal 30 transmits a registration request of theretrieval target image to the image retrieval server 20 (step S1). Thisregistration request includes a retrieval target image, contentinformation, and a destination address for transmitting data to theregistration terminal 30. A destination address for identifying thedestination to which the registration terminal 30 transmits theregistration request is set in advance in a program executed by theregistration terminal 30, for example. The controller 21 of the imageretrieval server 20 having received the registration request through thecommunication unit 22 executes registration processing (step S2).

FIG. 8 is a flowchart illustrating the flow of registration processing.FIGS. 9A through 9C illustrate a specific example of registrationprocessing.

The controller 21 of the image retrieval server 20 obtains a retrievaltarget image included in the registration request (step S21). Then, thecontroller 21 extracts specifying information from the obtainedretrieval target image (step S22). It is now assumed that a retrievaltarget image Is of FIG. 9A is obtained by the controller 21. In thiscase, as illustrated in FIG. 9B, the controller 21 performs imageprocessing for decoding information from a digital watermark dw1 on theretrieval target image Is, and thereby extracts specifying information.The digital watermark dw1 is added to the retrieval target image by anoperation performed by the registration terminal 30. In this example,the controller 21 extracts specifying information “G001”.

Then, the controller 21 detects feature points from the obtainedretrieval target image, and extracts the local feature amounts of thedetected feature points (step S23). In this example, as illustrated inFIG. 9C, the controller 21 detects feature points ps (indicated by blackcircles) such as edges and corners of the retrieval target image Is, andextracts the local feature amounts of the detected feature points ps.For simplicity of illustration, only one feature point is denoted by“ps” in FIG. 9C.

Not that although the digital watermark dw1 is superimposed on theretrieval target image, its effect on detection of feature points andthe local feature amounts may be ignored.

Then, the controller 21 registers, in the image retrieval database 231,the local feature amounts of the respective feature points extracted inthe processing of step S23 (step S24). In this step, the controller 21registers an image identifier and the local feature amounts, inassociation with the specifying information extracted in the processingof step S22. More specifically, the controller 21 assigns an imageidentifier to the retrieval target image Is obtained in the processingof step S21, and registers plural local feature amounts extracted fromthe retrieval target image Is. Referring to FIG. 5, each of the localfeature amounts “Vs1”, . . . , and “Vsk” associated with the specifyinginformation “G001” and the image identifier “IMG-1” is the local featureamount extracted from the retrieval target image with the imageidentifier “IMG-1”. In the processing of step S24, the controller 21also registers, in the image retrieval database 231, content informationincluded in the registration request.

The above is the description of the registration processing.

The image retrieval server 20 transmits the retrieval target image Isfor which the registration processing is performed to the registrationterminal 30, using the destination address transmitted in the processingof step S1. The registration terminal 30 generates a printed material 40by printing and outputting the retrieval target image Is. The imageretrieval server 20 executes registration processing with the proceduredescribed above, each time a registration request of a retrieval targetimage is received.

The discussion returns to FIG. 7.

It is assumed that the user of the user terminal 10 captures an image ofthe printed material 40 using the user terminal 10. In this case, thecontroller 11 of the user terminal 10 causes the image capturing unit 14to capture an image of the printed material 40, and obtains the imagecaptured by the image capture (step S3). Then, the controller 11transmits the obtained query image to the image retrieval server 20through the communication unit 13 (step S4). The destination address foridentifying the destination to which the user terminal 10 transmits thequery image is set in advance in a program executed by the controller 11of the user terminal 10, for example. Further, in the processing of stepS4, the user terminal 10 also transmits to the registration terminal 30a destination address to which data is transmitted. The controller 21 ofthe image retrieval server 20 having received the query image throughthe communication unit 22 executes retrieval processing on the basis ofthe query image (step S5).

FIG. 10 is a flowchart illustrating the flow of retrieval processing.FIGS. 11A through 11C illustrate a specific example of retrievalprocessing.

First, the controller 21 receives a query image (step S51). Then thecontroller 21 detects feature points from the obtained query image, andextracts the local feature amounts of the detected feature points (stepS52). It is now assumed that a query image Iq of FIG. 11A is obtained bythe controller 21. As illustrated in FIG. 11B, on the query image Iq,specifying information of this query image is superimposed as a digitalwatermark dw2. In this case, as illustrated in FIG. 11C, the controller21 extracts the local feature amounts of the feature points pq from thequery image Iq. For simplicity of illustration, only one feature pointis denoted by “pq” in FIG. 11C.

Then, the controller 21 extracts specifying information from theobtained query image (step S53). In this example, the controller 21performs image processing for decoding information from the digitalwatermark dw2 on the query image Iq, and thereby extracts specifyinginformation. This image processing may be the same as that performedwhen extracting the specifying information from the retrieval targetimage. In this example, the controller 21 extracts specifyinginformation “G001”.

Then, the controller 21 obtains the local feature amounts to be used forretrieval processing from the image retrieval database 231, and comparesthe obtained local feature amounts with the local feature amounts of thequery image (step S54). The controller 21 compares the local featureamounts of the feature points pq with the respective local featureamounts that are registered in the image retrieval database 231 inassociation with the specifying information extracted in the processingof step S53. As illustrated in FIG. 11C, the controller 21 obtains thelocal feature amounts corresponding to each of the image identifiers“IMG-1”, “IMG-2”, . . . , “IMG-N” associated with the specifyinginformation “G001”, and compares the local feature amounts of thefeature points pq with the obtained local feature amounts. That is, thecontroller 21 selects an image group specified by the specifyinginformation “G001” as a retrieval target. On the other hand, thecontroller 21 does not use, in the retrieval processing, the localfeature amounts (that is, an image group) associated with specifyinginformation, such as specifying information “G002” and “G003”, otherthan the specifying information extracted in the processing of step S53(for example, does not obtain from the image retrieval database 231).

Then, the controller 21 performs image retrieval processing thatretrieves a retrieval target image similar to the query image, on thebasis of the comparison result of step S54 (step S55). In this step, thecontroller 21 calculates the similarity with the local feature amountobtained in step S54, for each feature point pq of the query image Iq. Aknown algorithm may be used for calculating the similarity between thelocal feature amounts. The controller 21 determines corresponding pointsin the retrieval target image on the basis of the calculated similarity,and thereby retrieves a retrieval target image similar to the queryimage. For example, the controller 21 retrieves a retrieval target imagehaving the greatest number of corresponding points and having the numberof corresponding points greater than a threshold.

The controller 21 obtains content information associated with the imageidentifier of the retrieved retrieval target image, from the imageretrieval database 231 (step S56). For example, in the case where theretrieval target image with the image identifier “IMG-1” is theretrieval result, the controller 21 obtains content information “Ct-1”.

The above is the description of the retrieval processing. The imageretrieval server 20 performs the processing with the procedure describedabove, each time a query image is received.

The discussion returns to FIG. 7.

When the retrieval processing ends, the controller 21 of the imageretrieval server 20 transmits and outputs the content informationobtained in the processing of step S56 to the user terminal 10 throughthe communication unit 22, on the basis of the destination addresstransmitted in the processing of step S4 (step S6). The controller 11 ofthe user terminal 10 having received the content information through thecommunication unit 13 obtains and displays a digital content on thebasis of the received content information (step S7).

The above is the description of operations of the image retrieval system1.

In the image retrieval system 1 of the first exemplary embodimentdescribed above, the image retrieval server 20 extracts specifyinginformation that specifies an image group as a retrieval target from aquery image, in retrieval processing. Then, the image retrieval server20 obtains, from the image retrieval database 231, local feature amountsof each retrieval target image corresponding to the specifyinginformation, and compares the obtained local feature amounts with thelocal feature amounts of the query image. Accordingly, in the imageretrieval system 1, image retrieval is performed only on retrievaltarget images on the basis of the specifying information, withoutrequiring the user of the user terminal 10 to specify an image group asa retrieval target (retrieval target images). Since image retrieval isperformed only on the specified retrieval target images, the retrievalresult is not obtained from an image group corresponding to differentspecifying information. Further, the processing amount of the retrievalprocessing is reduced.

Second Exemplary Embodiment

Next, a description will be given of a second exemplary embodiment ofthe present invention.

In an image retrieval system 1 of the second exemplary embodiment,specifying information is extracted from a query image on the basis ofan image of an object, such as an icon image, that is visuallyrecognized by humans.

The apparatuses included in the image retrieval system 1 of the secondexemplary embodiment and the hardware configuration of the apparatusesmay be the same as those of the first exemplary embodiment. Further, thefunctional configuration of the image retrieval system 1 of the secondexemplary embodiment may be the same as that of the first exemplaryembodiment, except for a part related to the function for extractingspecifying information.

In the following description, the same parts as those of the firstexemplary embodiment will not be further described. Further, the sameelements and processing steps as those of the first exemplary embodimentare denoted by the same reference numerals, and will not be furtherdescribed.

FIGS. 12A and 12B illustrate a method by which the image retrievalsystem 1 extracts specifying information according to this exemplaryembodiment. FIG. 12A illustrates an example of a printed material 40according to this exemplary embodiment. As illustrated in FIG. 12A, inthe printed material 40, an icon image 414 indicating specifyinginformation is contained inside a feature boundary 411. In this example,the icon image 414 is a rectangular image with a star inside, and is animage (for example, the logo of a content provider or the like) thatrepresents the specifying information. Referring to FIG. 12B, a queryimage Iq1 obtained by capturing an image of the printed material 40 isillustrated. Since the icon image 414 is located inside the featureboundary 411, an object image A obtained by capturing an image of theicon image 414 is contained in the query image Iq1.

In the case where the query image Iq1 of FIG. 12B is obtained in theprocessing of step S51 of retrieval processing, the controller 21 of theimage retrieval server 20 detects an object image indicating thespecifying information from the query image Iq1 in the processing ofstep S53. The controller 21 stores the specifying information table 232of FIG. 12C in the storage unit 23, and detects an object imagecontained in the query image on the basis of the specifying informationtable 232. The specifying information table 232 is a data table in whichspecifying information and an object image that uniquely identifies thespecifying information are registered in association with each other.The controller 21 detects an object image registered in the specifyinginformation table 232 from the query image Iq1, using a known methodsuch as pattern matching, for example. In this step, if a mark imageserving as a mark of the object image (for example, a rectangular imagedescribed above) is contained in the printed material 40, the controller21 may detect the object image by recognizing the mark image.

Then, the controller 21 extracts specifying information associated withthe object image that is detected from the query image Iq1, on the basisof the specifying information table 232. The controller 21 performs theprocessing of step S54 of the retrieval processing and that follows, onthe basis of the extracted specifying information.

In the registration processing, the image retrieval server 20 uses aretrieval target image containing an object image that is uniquelyassociated with specifying information. Accordingly, the registrationterminal 30 transmits a retrieval target image containing an objectimage corresponding to specifying information specified by the user tothe image retrieval server 20. The image retrieval server 20 havingreceived the retrieval target image detects the object image containedin the retrieval target image on the basis of the specifying informationtable 232, and extracts the specifying information. After the specifyinginformation is extracted, the image retrieval server 20 performsregistration processing in the same manner as the first exemplaryembodiment.

Variations of Second Exemplary Embodiment

The variations described below may be appropriately combined with eachother.

(1) In the image retrieval system 1 of the second exemplary embodimentdescribed above, in the case where a feature point is detected from anobject image contained in a query image, the retrieval processing may beperformed without using this feature point. Thus, in the image retrievalsystem 1, the image retrieval processing is executed without beingaffected by the icon image 414.

It is assumed that the controller 21 detects the object image A from thequery image Iq1 as illustrated in FIG. 12B, and also detects a featurepoint pqj representing the feature of the object image A as illustratedin FIG. 13. In this case, the controller 21 determines correspondingpoints for the feature points that are detected from the area excludingthe object image A of the query image Iq1, on the basis of the localfeature amounts in the image retrieval database 231. On the other hand,the controller 21 does not use the feature point pqj detected from theobject image A in the image retrieval processing, and does not determineits corresponding point. The controller 21 performs image retrievalprocessing on the basis of the local feature amounts of the featurepoints detected from the area excluding the object image A. Theoperations of the image retrieval system 1 in step S55 of the imageretrieval processing and that follow may be the same as those of thesecond exemplary embodiment described above.

When performing registration processing, the image retrieval server 20may register, in the image retrieval database 231, the local featureamounts of the feature points detected from the area excluding theobject image in the retrieval target image.

(2) In the image retrieval system 1 of the second exemplary embodimentdescried above, in the case where an image variation (that is, noise)due to the way that the user terminal 10 captures an image of theprinted material 40 or the environment in which the image is captured iscontained in the query image, detection of feature points and extractionof local feature amounts may be performed after performing imageprocessing that reduces image variation on the query image. This imagevariation is an image distortion in the query image due to the way ofcapturing an image, for example. If the user terminal 10 is inclined inthe horizontal direction when an image is captured using a camerafunction, the obtained captured image might be distorted into aparallelogram shape or a trapezoidal shape in the horizontal direction.If the image capturing device is inclined in the vertical direction, theobtained captured image might be distorted into a parallelogram shape ora trapezoidal shape in the vertical direction. The feature points andthe local feature amounts detected from the query image may varydepending on the presence or absence of this type of image distortionand the degree of image distortion. If an image distortion occurs in thequery image, the accuracy of the image retrieval processing might bereduced.

An image retrieval server 20 of this variation realizes a functioncorresponding to the image processing unit 208, in addition to thefunctional configuration described in FIG. 6. The image processing unit208 performs image processing on the query image such that the objectimage detected from the query image becomes close to the object image ofthe specifying information corresponding to the object image registeredin the specifying information table 232. The feature informationextracting unit 202 extracts feature points and extracts the localfeature amounts, on the basis of the query image after the imageprocessing by the image processing unit 208.

FIG. 14 is a flowchart illustrating the flow of retrieval processingexecuted by the image retrieval server 20.

The controller 21 of the image retrieval server 20 detects an objectimage from the query image obtained in the processing of step S51. Then,the controller 21 compares the detected object image with the objectimage registered in the specifying information table 232, and specifiesan image variation in the query image. In the case of an imagedistortion, the direction and the degree of the image distortion may bedetermined by referring to the size and shape of the object image in thequery image. It is assumed that, as illustrated in FIG. 15A, a queryimage Iq2 with a trapezoidal image distortion in the horizontaldirection is obtained. In this case, in the object image A differs fromthe object image registered in the specifying information table 232, andis deformed due to the image distortion. Accordingly, the controller 21performs image processing (geometric transformation) on the query imagesuch that the detected object image becomes close to the object imageregistered in the specifying information table 232 (step S57). Thus, asillustrated in FIG. 15B, a query image Iqr with reduced image distortionis obtained. This query image Iqr contains an object image Ar whoseimage distortion is reduced by the image processing.

It is known that image retrieval using the SIFT feature amount is lesslikely to be affected by a reduction in retrieval accuracy due to thedifference in the size between a query image and a retrieval targetimage. Thus, the controller 21 may perform image processing on the queryimage such that the shape of the detected object image becomes close tothe shape (similar shape) of the object image registered in thespecifying information table 232.

Then, in the processing of step S52, the controller 21 extracts thelocal feature amounts from the query image after the image processing.Then, in the processing of step S53, the controller 21 extractsspecifying information from the query image after the image processing.After that, the controller 21 performs image retrieval processing inaccordance with the procedure described above in the second exemplaryembodiment.

In this variation, the local feature amount is extracted afterperforming image processing that reduces image variation of the queryimage on the basis of the object image. This configuration may beapplied to image retrieval other than image retrieval involvingextraction of specifying information without designation by the user,for example.

Further, in this variation, image distortion is described as imagevariation in the query image. However, image variation is not limited toimage distortion. For example, in the case where the object image in thequery image is rotated by a predetermined angle with respect to theregistered object image, the image retrieval server 20 performs imageprocessing that rotates the query image by the same angle in the reversedirection, and then performs detection of feature points and extractionof local feature amounts. Further, in the case where a variation in thebrightness is determined to be present in the query image on the basisof the object image, the image retrieval server 20 performs imageprocessing that reduces the variation in the brightness in the queryimage, and then performs detection of feature points and extraction oflocal feature amounts.

(Variations)

The present invention may be implemented in the manner different fromthe exemplary embodiments described above. Further, the variationsdescribed below may be combined with each other.

(Variation 1)

The image retrieval system of each of the exemplary embodimentsdescribed above may be modified as described below.

FIG. 16 illustrates the overall configuration of an image retrievalsystem 1 a according to this variation.

As illustrated in FIG. 16, the image retrieval system 1 a is aninformation processing system that performs image retrieval, andincludes a user terminal 10, an image retrieval server 20, aregistration terminal 30, and registration servers 50 a, 50 b, and 50 c.The user terminal 10 and the registration terminal 30 have the sameconfiguration and perform the same operations as those of the exemplaryembodiments described above. The registration servers 50 a, 50 b, and 50c are registration apparatuses, each of which corresponds to differentspecifying information, and in each of which the local feature amountsand content information of retrieval target images included in an imagegroup specified by the corresponding specifying information areregistered. The specifying information is information that specifies aregistration server in which the local feature amounts of an image group(retrieval target images) specified by the specifying information areregistered.

The registration server 50 a has a database in which data associatedwith the specifying information “G001” in the image retrieval database231 of FIG. 5 is registered. The registration server 50 b has a databasein which data associated with the specifying information “G002” in theimage retrieval database 231 of FIG. 5 is registered. The registrationserver 50 c has a database in which data associated with the specifyinginformation “G003” in the image retrieval database 231 of FIG. 5 isregistered. The image retrieval server 20 communicates with the userterminal 10 and the registration terminal 30, and also communicates withthe registration servers 50 a, 50 b, and 50 c. That is, the imageretrieval server 20 serves as a server that mediates the communicationbetween the user terminal 10 and the registration terminal 30 and theregistration servers 50 a, 50 b, and 50 c. Hereinafter, the registrationservers 50 a, 50 b, and 50 c may be referred to collectively as“registration servers 50” when the registration servers 50 a, 50 b, and50 c do not need to be referred to individually.

In this variation, the registration server 50 may perform registrationprocessing, and register the local feature amounts and contentinformation in the database. Alternatively, the image retrieval server20 may perform registration processing, and register the local featureamounts and content information in the database of the registrationserver 50. Further, in this variation, three registration servers 50 a,50 b, and 50 c are provided. However, two or four or more registrationservers 50 may be provided.

The image retrieval server 20 does not include the image retrievaldatabase 231, but instead stores an apparatus management table 233 ofFIG. 17 in the storage unit 23. The apparatus management table 233 is adata table in which specifying information and an identifier(hereinafter referred to as an “apparatus identifier”) that uniquelyidentifies the registration server 50 are associated with each other. InFIG. 17, the apparatus identifiers are represented by the referencenumerals of the registration servers 50 a, 50 b, and 50 c. That is, inthe apparatus management table 233, the corresponding relationshipbetween the specifying information and the apparatus identifier thatidentifies the registration server 50 storing the local feature amountsof the image group specified by the specifying information is specified.

The apparatus identifier may be access destination information(communication address) for accessing each registration server 50.Examples of the access destination information include a uniformresource locator (URL) of the service involving retrieval processing,and destination information indicating the destination of a query image.

FIG. 18 is a sequence diagram illustrating the flow of retrievalprocessing according to this variation.

The controller 21 of the image retrieval server 20 performs processingof steps S51 through S53 so as to extract the local feature amounts froma query image and extract specifying information.

Then, the controller 21 determines the registration server 50corresponding to the specifying information extracted in the processingof step S53 by referring to the apparatus management table 233, andtransmits the local feature amounts of the query image to the determinedregistration server 50 (step S58). For example, if the specifyinginformation that is obtained on the basis of the query image is “G001”,the controller 21 requests the registration server 50 a to performretrieval processing.

The registration server 50 compares the local feature amounts receivedfrom the image retrieval server 20 with the local feature amountsregistered in the registration server 50 (step S54), and performs imageretrieval processing that retrieves a retrieval target image similar tothe query image (step S55). That is, the registration server 50 of thisvariation serves as an image retrieval apparatus (image retrievalserver) of each of the exemplary embodiments of the present invention.

The registration server 50 having retrieved the retrieval target imagesimilar to the query image obtains content information associated withthe image identifier of the retrieved retrieval target image from thedatabase, and transmits the content information to the image retrievalserver 20 (step S59). For example, in the case where the registrationserver 50 determines the image identifier “IMG-1” as the retrievalresult, the registration server 50 transmits content information “Ct-1”associated with the image identifier “IMG-1” to the image retrievalserver 20. The controller 21 of the image retrieval server 20 obtainsthe content information from the registration server 50, and then theretrieval processing ends (step S56).

In the image retrieval system 1 a of this variation, the image retrievalserver 20 may obtain the local feature amounts from the registrationserver 50 and perform retrieval processing. In this case, the imageretrieval server 20 transmits the image identifier as the result of theretrieval processing to the registration server 50, and obtains contentinformation associated with the image identifier from the registrationserver 50.

(Variation 2)

In the image retrieval system 1 a of the variation 1 described above,each application program may correspond to different specifyinginformation. In this case, for example, an identifier (applicationidentifier) that uniquely identifies an application program may be usedas specifying information.

As for operations of the registration terminal 30, in the case ofexecuting an application program and requesting the image retrievalserver 20 to perform registration processing, the registration terminal30 transmits specifying information obtained from the application to theimage retrieval server 20 in the processing of step S1. As foroperations of the user terminal 10, in the case of executing anapplication program and requesting the image retrieval server 20 toperform retrieval processing, the controller 11 of the user terminal 10transmits specifying information obtained from the application programto the image retrieval server 20 in the processing of step S4.

In the case where the configuration of this variation is applied to theabove-described variation 1, even if each provider of applicationprograms or each provider of services based on application programscorresponds to a different registration server 50, the registrationserver 50 corresponding to the application program executed by the userterminal 10 is able to provide services involving retrieval processing.

In the image retrieval system 1 a of this variation, the specifyinginformation does not need to be contained in the retrieval target imageor the query image (that is, the printed material 40).

(Variation 3)

The image retrieval system 1 a of the variation 1 or the variation 2described above may be modified as follows. In this variation, theregistration server 50 extracts the local feature amounts from a queryimage, and performs retrieval processing.

Referring to FIG. 19, the controller 21 of the image retrieval server 20obtains a query image from the user terminal 10 (step S51), and extractsspecifying information from the obtained query image (step S53). Then,the controller 21 transmits (forwards) the query image to theregistration server 50 corresponding to the specifying information, onthe basis of the extracted specifying information and the apparatusmanagement table 233 (step S60). The registration server 50 extracts thelocal feature amounts from the query image received from the imageretrieval server 20 (step S52), compares the extracted local featureamounts with the local feature amounts registered in the registrationserver 50, and performs image retrieval processing (steps S54 and S55).Then, the registration server 50 obtains content informationcorresponding to the result of image retrieval processing from thedatabase, and transmits the content information to the image retrievalserver 20 (step S61). The controller 21 obtains the content informationfrom the registration server 50 (step S56), and then the retrievalprocessing in the image retrieval system 1 a ends.

According to the image retrieval system 1 a of this variation, theprocessing amount of retrieval processing in the image retrieval server20 is reduced compared to the case of the variation 1 or the variation 2described above.

(Variation 4)

In the image retrieval system, the user terminal 10 may extractspecifying information from a query image, and transmit the specifyinginformation to the image retrieval server 20, together with the queryimage. In this case, the image retrieval server 20 performs retrievalprocessing using the specifying information received from the userterminal 10.

Alternatively, the user terminal 10 may extract the local featureamounts from a query image, and transmit the local feature amounts tothe image retrieval server 20. In this case, the image retrieval server20 performs retrieval processing using the local feature amountsreceived from the user terminal 10.

Further, the registration terminal 30 may transmit specifyinginformation that specifies this retrieval target image as a retrievaltarget to the image retrieval server 20, separately from the retrievaltarget image. In this case, the image retrieval server 20 obtains thespecifying information received from the registration terminal 30, andperforms registration processing.

(Variation 5)

In the image retrieval system 1, the local feature amounts extractedfrom a partial image forming a part of a retrieval target image or aquery image may be used as specifying information. Hereinafter, adescription will be given of operations of the image retrieval system 1in the case where the registration terminal 30 and the user terminal 10extract the local feature amounts and specifying information from aquery image.

Upon requesting the image retrieval server 20 to perform registrationprocessing, the registration terminal 30 first specifies a partial imageforming a part of a retrieval target image. For example, as illustratedin FIG. 20A, the registration terminal 30 divides the image area of aretrieval target image Is into quarters in a matrix form of 2 by 2, andspecifies an image area located in a predetermined position (in thisexample, the lower right image area) as a partial image Isp. Then, theregistration terminal 30 transmits, to the image retrieval server 20,the local feature amounts extracted from the partial image Isp asspecifying information, in the processing of step S1. In theregistration processing in the processing of step S2, the imageretrieval server 20 registers the local feature amounts transmitted asspecifying information from the registration terminal 30 in the imageretrieval database 231. In this example, as illustrated in FIG. 20B, theregistration terminal 30 registers, in the image retrieval database 231,the local feature amounts extracted from the partial image Isp asspecifying information.

Next, operations of the image retrieval system 1 in the retrievalprocessing will be described. FIG. 21 is a sequence diagram illustratingthe flow of the operations.

The controller 11 of the user terminal 10 causes the image capturingunit 14 to capture an image of the printed material 40, and obtains theimage captured by the image capture (step S3). Then, the controller 11extracts the local feature amounts from the query image, on the basis ofthe captured image (step S52). In this example, the query image Iq ofFIG. 11A is obtained by the controller 11. Then, the controller 11extracts, as specifying information, the local feature amounts from apartial image forming a part of the query image (step S53 a). In theprocessing of step S53 a, the controller 11 specifies, in the queryimage, a partial image located in a position corresponding to theposition of the partial image of the retrieval target image. In thisexample, as in the case of the registration terminal 30, the controller11 divides the image area of the query image into quarters in a matrixform of 2 by 2, and specifies an image area located in a predeterminedposition (in this example, the lower right image area) as a partialimage Iqp (see FIG. 20C).

In the processing of step S53 a, the controller 11 may specify thepartial image of the query image after determining the verticaldirection of an image element (that is, an image element of the capturedimage of the printed material 40) contained in the query image. This isbecause, the image area specified as a partial image differs between thecase as illustrated in FIG. 20C where an image of the printed material40 is captured with the vertical direction of the image elementcontained in the query image aligned with the vertical direction of thequery image and the case where, as indicated by the dotted area in FIG.20D, an image of the printed material 40 is captured with the verticaldirection of the image element contained in the query image Iq3 notaligned with the vertical direction of the query image Iq3 (in thisexample, in a state in which the image element contained in the queryimage is rotated by 90 degrees to the right).

Thus, for example, the controller 11 detects, from a query image,information (direction information) specifying the vertical direction ofan image element contained in the query image, such as characters andthe marker 41 (for example, the media icon 413) contained in the queryimage. Then, the controller 11 rotates the query image such that thevertical direction of the image element contained in the query imagecoincides with the vertical direction of the query image, on the basisof the detected direction information. The controller 11 specifies apartial image from the rotated query image, and extracts specifyinginformation. Alternatively, the controller 11 may specify, as a partialimage, an image area that is expected to be located in a predeterminedposition in the case where the vertical direction of the image elementcontained in the query image is aligned with the vertical direction ofthe query image, without actually rotating the query image. In this way,even in the case where the vertical direction of the image elementcontained in the query image Iq3 does not coincides with the verticaldirection of the query image Iq3, the controller 11 is able toaccurately specify the partial image Iqp as illustrated in FIG. 20D.

Note that the direction information described above may be anyinformation as long as the direction information indicates the shiftamount between the vertical direction of the query image and thevertical direction of the image element contained in the query image.

Then, the controller 11 transmits the local feature amounts extractedfrom the query image and the specifying information indicating the localfeature amounts of the partial image to the image retrieval server 20through the communication unit 13 (step S4 a). The controller 21 of theimage retrieval server 20 having received the local feature amountsextracted from the query image and the specifying information throughthe communication unit 22 executes image retrieval processing (steps S54and S55). The flow of image retrieval processing may be the same as thatof the first exemplary embodiment, except that the local feature amountsof the partial image is used as specifying information. Then, thecontroller 21 obtains content information corresponding to the result ofimage retrieval processing from the database, and transmits the contentinformation to the user terminal 10 (steps S56 and S6).

According to the image retrieval system 1 of this variation, thespecifying information does not need to be contained in the retrievaltarget image or the query image.

The configuration that uses the local feature amounts extracted from apartial image of a query image as specifying information may be appliedto the case where an apparatus other than the user terminal 10 extractsthe local feature amounts and specifying information. Further, the sizeand shape of the partial image and the position of the partial image inthe retrieval target image or in the query image are not limited tothose described in the above example. For example, the conditions forspecifying a partial image may be determined such that partial images oftwo or more retrieval target images have common local feature amounts.

(Variation 6)

The specifying information of a retrieval target image or a query imagein the exemplary embodiments of present invention does not have tocorrespond to a content provider. The specifying information may beinformation indicating the type of service provided by a contentprovider, or information indicating the date and time when an image of aquery image is captured or the date and time when a retrieval targetimage is registered, for example. Further, the specifying informationmay be the application identifier described above.

That is, the specifying information in the exemplary embodiments of thepresent invention may be any information as long as the specifyinginformation is information used for specifying an image group as aretrieval target. According to the image retrieval system of thisvariation, regardless of the specifying information of the query imageand retrieval target image, retrieval target images used in theretrieval processing are selected without requiring the user to specifyan image group as a retrieval target.

(Variation 7)

The specifying information in the image retrieval database 231 does nothave to match the specifying information extracted from a query image.That is, in the image retrieval database 231, the local feature amountsand content information corresponding to a retrieval target image may beregistered without using specifying information of a query image. Inthis case as well, the image retrieval server 20 obtains the localfeature amounts of each retrieval target image corresponding tospecifying information extracted from a query image, and performsretrieval processing.

For example, in the case where query images are classified into smallercategories than retrieval target images, the image retrieval server 20may use, in retrieval processing, the local feature amounts of retrievaltarget images in a category containing a category of a query image.Conversely, in the case where query images are classified into largercategories than retrieval target images, the image retrieval server 20may use, in retrieval processing, the local feature amounts of retrievaltarget images in a category contained in a category of a query image.Further, the image retrieval server 20 may determine, on the basis ofspecifying information, the local feature amounts of retrieval targetimages in a category which is semantically related to the category of aquery image, and use the determined local feature amounts in theretrieval processing.

(Variation 8)

In the image retrieval system 1 of the second exemplary embodimentdescribed above, an object image uniquely associated with specifyinginformation is not limited to an icon image, but may be an encoded imagecontaining encoded information such as a barcode and a two-dimensionalcode. Further, the object image may be a character string of one or morecharacters that uniquely identifies the specifying information. That is,information indicating the specifying information of a query image iscontained in the query image.

Further, in the image retrieval system 1 of the second exemplaryembodiment described above, an object image that is uniquely associatedwith specifying information may be used in place of the media icon 413.

(Variation 9)

In the exemplary embodiments described above, a display apparatus thatdisplays an image containing a marker 41 superimposed on the imageelement may be used in place of the printed material 40. In this case,the user terminal 10 captures an image (screen) displayed by the displayapparatus.

(Variation 10)

In the exemplary embodiments described above, the SIFT feature amount isused as the feature amount (local feature amount) of a feature point.However, other types of feature amounts, such as the feature amountbased on speeded up robust features (SURF) may be used. That is, thefeature information in the exemplary embodiments of the presentinvention is not limited to the features of the intensity gradients, butmay be information that indicates the location of a feature point(coordinates) and the feature amounts of other feature points.

Further, the feature information in the exemplary embodiments of thepresent invention may indicate the feature amounts that may be used forimage retrieval, other than the local feature amounts of feature points.

(Variation 11)

In the exemplary embodiments described above, the image retrieval server20 outputs content information corresponding to a retrieval target imagesimilar to a query image. However, information that is output is notlimited to content information. For example, in the case where thepresent invention is applied to an information processing system thatretrieves a similar image, the image retrieval server 20 may transmitand output image data of a similar image retrieved from the retrievaldatabase to the user terminal 10. In this case, a retrieval target imageidentified by an image identifier may be registered in the imageretrieval database 231, in place of content information.

(Variation 12)

The image retrieval server 20 does not need to have a function forregistering the local feature amounts in the image retrieval database231. In this case, an image registration apparatus separately providedfrom the image retrieval server 20 may perform registration processingthat registers the local feature amounts and content information in theimage retrieval database 231.

Further, the image retrieval server 20 may be realized not by a singleserver apparatus, but by plural server apparatuses that operate incombination. Further, the functions realized by the image retrievalserver 20 may be realized by a communication terminal (for example, theuser terminal 10) that is used by the user.

The functions realized by the user terminal 10, the image retrievalserver 20, the registration terminal 30, and the registration server 50of each of the exemplary embodiments described above may be realized byone or more hardware circuits, by execution of one or more programs byan arithmetic device, or by combination of these. In the case where thefunctions of the user terminal 10, the image retrieval server 20, theregistration terminal 30, and the registration server 50 are realized byusing a program, the program may be stored in the form of being storedin a computer-readable recording medium such as a magnetic recordingmedium (magnetic tape, magnetic disk, hard disk drive (HDD), flexibledisk (FD), and so on), an optical recording medium (optical disk and soon), a magneto-optical recording medium, and a semiconductor memory, ormay be delivered over a network. Further, the present invention may beembodied as an information processing method performed by a computer.

The foregoing description of the exemplary embodiments of the presentinvention has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit theinvention to the precise forms disclosed. Obviously, many modificationsand variations will be apparent to practitioners skilled in the art. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, therebyenabling others skilled in the art to understand the invention forvarious embodiments and with the various modifications as are suited tothe particular use contemplated. It is intended that the scope of theinvention be defined by the following claims and their equivalents.

What is claimed is:
 1. An image retrieval system comprising: anobtaining unit that obtains a query image; a specifying informationextracting unit that extracts, from the query image, specifyinginformation which specifies an image group as a retrieval target; afeature information extracting unit that extracts, from the query image,feature information to be used in image retrieval processing; and aretrieval unit that performs the image retrieval processing on retrievaltarget images specified by the specifying information, using the featureinformation.
 2. The image retrieval system according to claim 1, whereinthe specifying information extracting unit extracts the specifyinginformation, by comparing an object image detected from the query imagewith an object image which is registered in advance in association withthe specifying information.
 3. The image retrieval system according toclaim 2, wherein the feature information includes feature amounts offeature points; and wherein the retrieval unit performs the imageretrieval processing, without using the feature amounts of the featurepoints contained in the detected object image, the feature pointscontained in the detected object image being included in the featurepoints contained in the query image.
 4. The image retrieval systemaccording to claim 2, further comprising: an image processing unit thatperforms image processing on the query image such that the detectedobject image becomes close to the registered object image; wherein thefeature information extracting unit extracts the feature informationfrom the query image after the image processing.
 5. The image retrievalsystem according to claim 1, wherein the specifying informationextracting unit extracts the specifying information added to the queryimage as a digital watermark.
 6. The image retrieval system according toclaim 1, wherein the specifying information extracted by the specifyinginformation extracting unit is the feature information extracted from apartial image forming a part of the query image.
 7. The image retrievalsystem according to claim 1, further comprising: a plurality ofregistration apparatuses, each of which corresponds to differentspecifying information, and in each of which the feature information ofthe image group specified by the corresponding specifying information isregistered; wherein the retrieval unit performs the image retrievalprocessing, using the feature information registered in the registrationapparatus corresponding to the specifying information extracted by thespecifying information extracting unit.
 8. The image retrieval systemaccording to claim 1, further comprising: a registration processing unitthat registers the feature information; wherein the feature informationextracting unit extracts the feature information from the retrievaltarget image that is requested to be registered; wherein the specifyinginformation extracting unit extracts the specifying information from theretrieval target image that is requested to be registered; and whereinthe registration processing unit registers, as a retrieval target in theimage retrieval processing, the feature information of the retrievaltarget image extracted by the feature information extracting unit, inaccordance with specifying information of the retrieval target imageextracted by the specifying information extracting unit.
 9. The imageretrieval system according to claim 1, further comprising: a query imagereceiving unit that receives the query image transmitted from acommunication terminal; and an output unit that outputs, when the imageretrieval processing is performed on the basis of the query imagereceived by the query image receiving unit, content informationassociated with a retrieval target image retrieved by the imageretrieval processing to the communication terminal.
 10. An informationprocessing apparatus for use in an image retrieval system, wherein theimage retrieval system includes an obtaining unit that obtains a queryimage, a specifying information extracting unit that extracts, from thequery image, specifying information which specifies an image group as aretrieval target, a feature information extracting unit that extracts,from the query image, feature information to be used in image retrievalprocessing, and a retrieval unit that performs the image retrievalprocessing on retrieval target images specified by the specifyinginformation, using the feature information, the information processingapparatus comprising: the specifying information extracting unit.
 11. Animage retrieval method comprising: obtaining a query image; extracting,from the query image, specifying information which specifies an imagegroup as a retrieval target; extracting, from the query image, featureinformation to be used in image retrieval processing; and performing theimage retrieval processing on retrieval target images specified by thespecifying information, using the feature information.
 12. Anon-transitory computer readable medium storing a program causing acomputer serving as an information processing apparatus in an imageretrieval system to function as a specifying information extractingunit, wherein the image retrieval system includes an obtaining unit thatobtains a query image, the specifying information extracting unit thatextracts, from the query image, specifying information which specifiesan image group as a retrieval target, a feature information extractingunit that extracts, from the query image, feature information to be usedin image retrieval processing, and a retrieval unit that performs theimage retrieval processing on retrieval target images specified by thespecifying information, using the feature information.